1. Copyright Notice
A portion of this patent document contains materials, which are subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark office patent file or records, but otherwise reserves all copyright rights whatsoever.
2. Field of the Invention
Generally, the present invention relates to the monitoring and analysis of the behavioral phenotype of targets, such as vertebrates (e.g. zebra fish, or Danio rerio, More specifically, one aspect of the invention is directed to the automatic monitoring and analysis of the 3-D motion-related behavior of laboratory animals, such as locomotion activity, motor activity, home cage behavior, aggression, antipredatory, group preference, and light preference paradigms, under specific behavioral paradigm experiments, for a individual animal, or a group (school) of animals, in either real-time and on-line mode or off-line mode. The laboratory animals can be genetically altered animals after knock-in, knock-out, or transgenic operation, or experimental animals after being exposed to drugs, chemicals or certain environments.
3. Related Art
Experimental laboratory animals such as wild-type animals, genetically altered (transgenic, knock-in, or knock-out) animals, drug-treated animals, and animals after chemical mutagenesis operations, have been extensively used as human models in various biological, clinical, biomedical, environmental, and military research areas, including genomic research, studies of genetic functional disorders, phenotypic drug screening, toxicology, bio-sensing, and bio-defense, just to name a few. This is due to the fact that humans and animals share extensive genetic and neuroanatomical homologies, which are widely conserved among different species. The behavior studies of the animal models are particularly useful in post-genomic research areas towards identifying known genotypes, quantifying the behavior responses induced by various neurological disorders, and revealing the toxicity and efficacy of drug candidates.
Deficiencies in motor function may be caused by genetic mutations or by the effects of chemical compounds. Motion-related animal behavioral studies are useful in understanding the effects of different genotypes on the development of various motion-related functional diseases, such as Huntington's disease and Parkinson's disease, as well as the effects of drugs or any chemical compound on humans. Typical animal models used for these purposes include rodents such as mice and rats, vertebrates such as zebrafish and goldfish, and insects such as drosophilae. A variety of standardized animal behavioral tests have been designed with these models. For example, the behavioral tests for rodents are composed of open field, home cage, water maze, and social behavior paradigms, while the behavioral tests for vertebrates are composed of swimming locomotor activity, antipredatory behavior, and group preference paradigms. The key parameters describing the phenotypic behavior have been defined for these tests. For example, the swimming locomotive behavior of zebrafish can be classified and analyzed by traveling distance, swimming speed, turning angle, average rate of change of direction (RCDI), net to gross displacement ratio (NGDR), body wave shape, and tail beat amplitude and frequency, etc.
Among all the standardized behavioral tests for laboratory animals of various species, motion information and spontaneous activity information are of great importance for phenotypic screening. Such information can be obtained from open field (for locomotor activity test) and home cage paradigms.
The monitoring of motion patterns of laboratory animals has historically been accomplished by human observation and/or off-line manual counting on pre-recorded videotapes, which inevitably resulted in inaccurate, inadequate, and subjective data and observation results. Furthermore, human observation methods have significant drawbacks such as lacking of quantitative data, large observation variations, labor-intensiveness, high costs, and missing of information along the depth direction of human eyes. Recently, researchers have developed various computerized apparatuses and methods to automatically monitor the locomotion/motor behavior of animals, including photobeam cage, force actoplate, and 2-D video recording combining with off-line video sequence analysis, just to list a few. See, e.g., S. Kato, et al, A computer image processing system for quantification of zebrafish behavior, Journal of Neuroscience Methods, 134(2004), 1-7; and J. Chraskova, et al., An automatic 3-D tracking system with a PC and a single TV camera, Journal of Neuroscience Methods, 88(1999), 195-200. Among these methods, the video recording method has unique advantages over other methods, such as non-contact setup, high sampling frequency, high spatial resolution, long monitoring period, the ability of tracking the motion of specific parts of the body, and versatility in tracking the motion of different species. Therefore, the 2-D video recording and analysis method is more widely applied in the field of animal behavior monitoring and analysis.
However, there are still significant drawbacks associated with the existing 2-D video monitoring and analysis systems. For example, existing video monitoring systems typically collect those kinematical parameters describing animal planar motion only by using a single video camera, i.e., a horizontal plane if the camera views from the top of the motion field. The camera of existing 2-D video systems generally shoots a single view of the animal container thus losing information along other spatial axes, such as the camera axis perpendicular to the plane defined by the image plane of the camera. Consequently, existing 2-D video systems (e.g., the system described by S. Kato, et al., Journal of Neuroscience Methods, 134(2004), 1-7) generally can not detect the upward or downward motion of the tested animals, e.g., the rearing motion of mice and up-and-down swimming motion of zebrafish, because the camera usually shoots from the top of a mouse cage or fish tank. In addition, existing video tracking systems have limited capability in monitoring multiple moving animals residing in the same container and lose the motion information associated with certain animals if the animals being tracked are occluded by other animals, or if part of the animal body, which may be of interest, is occluded by the animal body itself. For example, the footpath of mice or rats may be inaccessible to the 2-D video tracking system if the camera shoots from the top of the arena. In addition, existing video monitoring systems generally do not correct for some physical errors or environmental changes, e.g., they do not address the measurement error associated with water refraction and reflection, which should be corrected for, when monitoring fish swimming motion.
In general, there has been an increase in demand for automatic phenotypic behavior monitoring systems in the past a few years, which can be utilized in various behavior tests of laboratory animals. Examples of automatic systems have been developed according to these needs include: photobeam cages, force plate actometers, and analog/digital video monitoring systems. See, e.g., the articles cited above. The application of these automatic monitoring systems has successfully solved most of the subjectiveness problems associated with the conventional methods of human observation, such as low accuracy, labor intensiveness, and the resultant data errors. Among these systems and methods, the use of a video camera as the motion sensor has provided the most powerful monitoring capabilities due to the high spatial resolution of the camera and its adeptness to various animal species and environment. However, many existing video systems are 2-D in nature and can only monitor the motion information along two translation axes and one rotation axis, i.e. three degrees of freedom (DOF) defining a planar motion. Consequently, the gathered time histories of animal motion parameters are incomplete when using these conventional 2-D video-tracking systems. Therefore, a more advanced video system that is able to truly monitor the motion-related behavior of laboratory animals in 3-D space is demanded.
Real-time 3-D systems have been described wherein two or more cameras are used to capture images. In addition, 3-D systems have been described that consist of a combination of a camera and two or more mirrored surfaces, resulting in non-conventional stereo pairs. See, Gluckman and Nayar, A Real-Time Catadioptric Stereo System Using Planar Mirrors, IUW, 1998; Lin, J. Yeh, M. Ouhyoung, Extracting 3-D facial animation parameters from multiview video clips, IEEE CGA, 22(6), 2002, 72-80; and J. Chraskova, et al., An automatic 3-D tracking system with a PC and a single TV camera, Journal of Neuroscience Methods, 88(1999), 195-200. The systems described by Nayar, Lin and Chraskova relate to the capture of a single stereo image. In their systems, however, systematic implementations of 3-D video tracking of animal motion are not adequately addressed, such as calibrating the system, dealing with measurement error and system noises induced by multiple media, tracking the motion of a single or multiple animals robustly without aliasing, and tracking multiple animals simultaneously without attaching visibly distinguishable tags. For example, although reflective mirror is employed in the 3-D animal tracking system described by Chraskova, the mirror is only set at approximate orientation and position while not further calibrated for its accurate geometric parameters. In their system implementations for tracking the swimming behavior of fish, the monitoring errors such as water refraction-induced distortion of stereo geometry is not corrected. In the system described by Chraskova, furthermore, a light emitting diode (LED) marker has to be carried by every animal being tracked, while multiple LED markers have to be activated in alternate frames (time-sharing regime) in the applications of monitoring multiple animals simultaneously. These requirements significantly increase both the technical challenge in the implementation of the behavioral experiments and the uncertainties in the behavior monitoring results.
Accordingly, there exists a need for improved systems and methods for 3-D monitoring and analyzing the motion behavior of one or more test animals.